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ABSTRACT 

A study attempted to determine .lOw students in 
literature-based reading programs compare to students in basal-based, 
skill-oriented reading programs in terms of their interest in 
reading. The study was carried out on a sample of 215 pup'ls in 9 
third and fifth grade classrooms from school districts in Eastern 
Kansas and Northeastern Oklahoma. Students' attitudes toward school 
were measured with the Elementary Reading Attitude Survey, a 
standardized, self-report, paper and pencil test normed on students 
in grades 1-6. There are 20 items on the survey which constitute the 
total reading score. Results showed that posttest scores were in all 
cases below pretest scores, probably because reading interest tends 
to drop off toward the end of the school year. In both third and 
fifth grades, scores on the posttest, while lower than the pretest, 
were still higher in classes using the literature-based approach than 
in those using the skills-based approach. Generally, fifth graders 
scored lower than third graders on the reading attitude tests. 
Further, results from this study suggest the literature-based 
approach is especially important for males, particularly those in 
fifth grade. (Contains 2 figures, a table of data, and 33 
references.) (TB) 
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Literature-Based Approach 2 

Abstract 

The purpose of this study was to analyze the reading attitudes of third and fifth grade male students 
who participated in a literature-based and skills-based reading program. The Elementary Reading 
Attitude SurvQ^ was administered on two occasions (fall and spring) to determine attitudinal 
differ«ices among the two groups. Results of the study indicated there was no main effect for type 
of instruction. Third grade males showed no preference for a reading program, while fifth grade 
males showed a slight preference for a literature-based reading program. Both third and fifth 
graders, male and females, showed a slight preference for a literature-based reading program. 
Also, data obtained showed a main effect existed for gender which indicated females had more 
positive attitudes towards reading than males despite the type of instruction used. 
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Literature-Based Approach 3 



THE EFFECT OF A LITERATURE-BASED APPROACH 
ON THE READING ATTITUDES OF MALE STUDENTS 
IN THE THIRD AND FIFTH GRADES 

A recent trend in educational strat^ies has been the whole language approach to teaching. 
Whole language is based on the socio-psycholinguistic theory that reading is learned in context 
(Anderson, 1984). Bergeron (1990) best defines whole language as "a concqH that embodies a 
philosophy of language devdopment as wdl as the instructicMial appx)aches ^bedded within, and 
supportive of, that philosophy. This concept includes the use of real literature and writing in the 
context of meaningful, functional, and cooperative experiences in ordo" to devdop in students 
motivation and interest in the process of learning." 

Whole language uses an integrative process that teaches language with all reading 
systems/skills intact. Childr^ learn to respond to literature across all areas of the curriculum, 
while developing naturally their own rules for fhe sound and syntactical systems of language 
(Goodman, 1989). In a whole language approach, students are encouraged to use their own 
experiences and knowledge to comprehend literature mstead of distracting them from the context 
with decoding skills (Shapiro & Riley, 1989; Watson, 1989). 

An aspect of the whole language approach is to center reading and reading activities around 
childr«i's literature books. Literature-based reading instructicHi involves the use of literature 
instead of textbooks in the teaching of reading, and utilizes combinations of teadier and student 
interactions >^'ith the selection so the student will devdqp into a thoughtful and proficient reader 
(Giddings, 1990). A holistic litoature-based classroom diminates the structural support and 
practice of basal reading systems and uses children's trade books as the cornerstone of the reading 
program (Giddings, 1990). The theoretical base for a literature-based reading program focuses 
largely on the whole language philosophy (Giddings, 1990). For the purpose of this study, whole 
language and literature-based reading instruction will be used interchangeably in discussing this 
methodology of reading. 

Since the emergence of the whole language approach, a debate between skills-based and 
holistic approaches has surfaced The research suggests that classrooms with rich language and 
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Literature- Based Approach 4 
concept environments fail to provide direct instruction in the application of decoding skills 
(Heymsfeld, 1989; Schickedanz, 1990; Shapiro & Riley, 1989). Shapiro and Riley (1989) 
described the debate between the two as "the necessity for students to master skUls before 
integration versus the students' use of their personal knowledge and concepts to assist in their 
comprdioision of material." 

The skills-based approach is one of the most common approaches to teaching reading in U. S. 
classrooms. A skills-based approach uses basal textbooks, worksheets, drills, and phonics for 
instruction. Instructional programs in the U.S. focus on "essential" reading skills, despite the fact 
these skiUs have no demonstratable relationship to learning how to read books (Bussis, 1982). 
Often instruction in reading focuses on the practice of reading skills for an undefined purpose and 
children begin to see these tasks as tedious and lacking a connection to life beyond the classroom 

(Hiebert & Colt, 1989). 

Surveys indicate more than 90% of teachers use basals to teach reading and some states 
r€^ffitheuseofbasals(Gursky, 1991). Advocates of literature-based reading instruction believe 
basal readers often aeate artificial language and mar the use of literature by gearing it to skill 
development (Giddings, 1990). It is interesting to note that Holland and Hall (1989) could not 
find two basal readers that present reading skills in the same sequence, which suggests no 
particular skills sequence is necessary for learning to read. 

Although basal readers have improved, the new basal selections are not necessarily more 
interesting than the old basals (McCarthy, et al., 1994). Most textbooks printed after 1990 have 
less adapted literature and vocabulary control and repetition is reduced, however, skills are stiU 
taught in a prescribed sequence (O'Neil, 1994). In a survey of 178 Australian students, Caimey 
( 1988) found student perceptions of reading associated with basals were negative. "Children do 
not see meaning as important when reading basals, and they do not find basals intrinsically 
interesting" (Caimey, 1985). Research by Taylor, Frye, and Gaetz (1990) based on performance 
on skill pretests suggest many elementary students may be able to skip approximately 70-90% of 
the skill activities in the basal reader in which they are placed for instruction. 

Research comparing the use of basals to the use of trade books in classrooms tends to favor 
authentic literature for student achievement and positive attitudes. In a study by Bader et al. 
(1987), the researchers found that second grade students using literature books showed more 
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improvement in reading and more studoit interest than those using basals, A quantitative study on 
research comparing whole language to basal readers found whole language and basals were equal 
in their effects on achievement with these excqptions: whole language is more effective in 
kindergarten, and it is most effective for teaching the functional aspects of reading (i.e., print 
concepts and expectations) than at mastering word recognition skills (Stahl & Miller, 1989). 

In her review of literature on child development and reading styles, Carbo (1987) indicates that 
young boys may be at a greater risk in reading programs with a strong emphasis on listening, 
worksheets, and phonics because they tend to have less-well developed auditory and verbal skills 
and taid to learn through kinesth^ic activities (Carbo, 1987). Research by Eiszler (1982) indicates 
fifth and sixth grade male students prefer tactile and kinesthetic styles of learning with less 
preference for teacher direction. Research on brain dominance (Sinatra, 1983) found male readers 
and poor readers were strong in right hemisphere functioning, which is the holistic and nonverbal 
sphere of information in the brain. The holistic approach provides for students with all learning 
styles because it teaches a whole to part relationship and relies on hands-on activities. For this 
reason, many researchers (Carbo, 1987; Catoe, 1992; Rdff, 1992; Sinatra, 1983) recommend the 
use of a holistic approach to reading. 

Studies have also been conducted on the effectiveness of reading and writing achievemait in a 
holistic approach. Varble (1990) analyzed the writing samples of second graders and sixth graders 
taught by teachm with the whole language or skills-based approaches. The criteria of correctness 
was dq>endent upon the quality of the content and the mastery of mechanics. Varble (1990) found 
no difference between the sixth grade students* writing samples; however, second grade students 
taught with a whole language approach had writing samples with better meaning and content but 
there was no difference in the use of mechanics between the students taught with the different 
approaches. In a study of first grade students, Holland and Hall (1989) found no significant 
difference in reading achievement between classes taught with a basal reader or a whole language 
approach. However, students taught with a whole language approach enjoyed reading and 
participated more in class than students taught with a basal. A study conducted by personnel in 
Franklin County School District, Meadville, Mississippi (1993) found reading achievement for 
first and second grade students in a whole language classroom well exceeded the national averages 
for reading. These students also reported the most positive attitudes towards their teachers. 
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Reutzel and Cooter ( 1990) also measured first grade reading achievement between basal readers 
and whole language and found a significant difference in a holistic approach ovct basal readers in 
total reading scores, and in vocabulary and comprehension subtests. 

In gender studies regarding reading instruction, Closer and Pearman (1992) administa-ed the 
Elementary Reading Attitude Survey developed by McKenna and Kear ( 1992) to grades 1 -6. 
Closer and Pearman ( 1992) found there were no significant differences between the recreational 
reading attitudes nor the acadwiic reading attitudes of boys and girls in grades 1-3. Howevo", the 
attitudes of girls towards academic reading in grades 4-6 were significantly lower than their 
recreational reading attitudes. Additionally, boys in grades 4-6 dropped significantly in both 
recreational and academic reading as compared to boys in grades 1-3. A review of the literature by 
Sheridan and Fizdale (1981) revealed children early in their school careers learn reading is closely 
identified with the female role. In a study on sex-role appropriateness on studoits in grades 
Shapiro ( 1 990) found half of the boys and girls near the end of kindergarten viewed reading and 
writing as a sex-role appropriate activity. Additionally, Shapiro (1990) noted boys at the end of 
first grade inaeasingly viewed reading and writing as feminine, and by second grade nearly all 
boys viewed writing as a feminine activity. 

"The attitudes of learners is crucially important to the outcomes from their woric" (Howard, 
1988). Attitudes towards school develop early, and the elementary years are important in 
developing good readers (Howard, 1988). Research by psychologist Benjamin Bloom (1976) 
found students' perception of success or failure in school learning tasks should result in favorable 
or unfavorable attitudes towards school and school learning. Further, Bloom ( 1976) noted the 
positive relation between affect toward school and school achievement. 

As the research has shown, many studies have focused on the reading achievement of primary 
students using a whole language, literature-based approach. However, few studies have analyzed 
reading attitudes in relation to male students. This study focused on the comparison between the 
reading attitudes of male students who were educated in a whole language environment to the 
reading attitudes of males who were educated in a skills-based environment. This study also 
compared the reading attitudes of males to females within the same grade and learning 
environment. 
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METHOD: 

Subjects 

This research study was carried out on a sample of 215 pupils in nine classrooms from school 
districts in Eastern Kansas and Northeastern Oklahoma. Four classrooms from fifth grade and five 
classrooms from third grade were chosen. All fifth grade classes were located in Miami, 
Oklahoma Public School District. Three of the third grade classes were located in Baxter Springs, 
Kansas Public School District, while the other two third grade classes were located in DeSoto, 
Kansas Public School District. The classrooms had an average of 24 students with approximately 
half of the students being male. The classrooms chosen for this study were selected on the 
building principals' evaluations of the teaching method used by each teacher, and were validated by 
thirty minute classroom observations conducted by the researcher. There was no random selection 
for this sample, as the clacNses in the schools were already de^mnined prior to the beginning of the 
study. 

Desig n 

A quasi-experimental study was conducted using these students. One indq)endent variable 
was whether or not a literature-based reading approach was used as the m^hod for reading 
instruction. Literature-based reading instruction is defined as a method of instruction that actively 
involves students in the reading process while teaching them to respond to literature across the 
curriculum. This literature-based approach took place in the form of reading instruction and 
activities. The dependent variable was the students* attitude towards reading. Students' attitudes 
towards reading was measured with the Elementary Reading Attitude Survey (McKenna & Kear, 
1990). 

A quasi-experimental pretest-posttest design was used for each grade level tested Raw 
scores for each student was utilized in the analyses. Mixed repeated measures analyses of 
covariance were computed to determine main effects and interaction effects of the independent 
variables and covariate. The ANCOVA design was a four-way treatment structure with three 
dependent variables. Males and females represented the two levels of ^gender'. Instruction with 
literature-based reading appro ich and instruction without literature-based reading approach 
rq)resented the two levels of instruction'. The two levels of 'grade' represented third and fifth 
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grades. Pretest and posttest represented the two levels of 'time of testing^ The dependent 
variables were the raw scores on the attitude measurement, scores from the two subtests and a total 
reading score. The covariate was the students' group achievement scores taken from the spring of 
the previous school year. The total reading national perc«itile rank scores from a standardized 
achievement test were used in the analyses. Scores from the Iowa Test of Basic Skills were 
utilized for the third grade students, and the Metropolitan Achievement Test - 7 scores were utilized 
for fifth grade students. 

Instrumentation 

Students' attitudes toward school were measured with the Elemaitary Reading Attitude 
Survey. The survey is a standardized, self-rqx)rt, paper and pencil test normed on students in 
grades 1-6. There are twenty items on the survey which constitute the total reading score. The 
first ten items represent the recreational reading score, the final ten items rq>res«it the academic 
reading score. Pretest and posttest data were reported as raw scores. This assessment instrument 
is supported by numerous validity and reliability studies (McKenna & Kear, 1990). The test was 
group administ^ed by the researcher. Fifth grade classrooms were pretested in October and 
posttested in April. The third grade classrooms were pretested in November and posttested in 
May. There was a time spread of six months b^ween the pretest and posttest. Teacher 
involvement was minimal. 

Procedures 

Instruction remained the same throughout the duration of the study with two classes from each 
grade level, third and fifth grades, receiving reading instruction using the literature-based 
approach. The other classes from each grade level received skills-based reading instruction using a 
basal reader as adopted by the individual school districts. 

Reading instruction with the literature-based approach focused on active reading and hands- 
on, discovery learning using real literature books. Reading lessons in all classrooms involved 
whole group instruction with one selected literature book, such as Sign of the Beaver or Mother 
Goose's Nursery Rhvmes. In each of the literature-based classrooms, the researcher observed 
many cooperative group activities, writing activities, and thematic units that aossed into other 
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areas of the curriculum. The researcher also noted more teacha involvement and interaction with 
the students. Each classroom met the criteria for "characteristics of a literature-based classroom" 
checklist developed by the researcher. 

Classes without the literature-based reading approach had reading instruction that focused on 
basal reading skills, phonics instruction, and direct, teachCT instruction. Matmals used in these 
classrooms included Hooked on Phonics, SRA reading series, and a 1991 Harcourt Brace & 
Janovich basal reading series. The researcho" observed "round robin" reading, worksheets and 
workbook activities, vocabulary instruction, and word attack skills. The researcher noted students 
were encouraged to "sound out" words when reading aloud Each classroom met the criteria for 
"characteristics of a skills-based classroom" checklist developed by the researcher. 

At the beginning and end of the school year, all groups were administered a survey measuring 
reading attitudes. The researcher administered the Elementary Reading Attitude Survey to all 
classrooms in a whole group format. Children were guaranteed anonymity and identified their 
gender with a "B" or "G" and a number at the top of their attitude survey. The researcher 
emphasized this was not a test and there were no "right" answers. All four Garfidds were 
discussed in a group format before beginning the survey. Students were encouraged to circle the 
Garfield that matched their feelings, not as Garfield might respond Each item was read aloud to 
the students by the researcher. For example, questions asked on the survey were "How do you 
feel about going to a bookstore?" and "How do you feel when the teacher asks you questions about 
what you read?" This same procedure was followed for both priest and posttest. 

RESULTS 

The dependent variables in the following analyses were the total reading attitude scores and the 
scores on the two subtests, recreational and academic reading attitude. These scores were entered 
into three separate 2 (Grade) x 2 (Gender) x 2 (Type of Instruction) x 2 (Time of Testing) repeated 
measures analyses of covariance, with repeated measures on time of testing (fall and spring testing) 
and with standardized achievement test scores as the covariate. 

The results with the total reading attitude scores indicated that there were main effects of grade, 
F(l, 143) = 24.50, e< -0001, gender, F(l, 143) = 12.81, ii< .0005, and time of testing, F(l, 
143) = 4.23, ut < .04, and an interaction of grade and time of testing, F ( 1 , 143) = 15.24, 2 < 



ERIC 



30 



Literature-Based Approach 10 
.0001. As can be seen in Figure 1 , females had higher total reading attitude scores than males. 

[Insert Figure 1] 

Inspection of Figure 2 reveals the interaction of grade and time of testing was due to the fifth- 
grade students' posttest total reading attitude scores being significantly lower than their pretest 
scores and significantly lower than third graders posttest scores, which were only slightly lower 
than their pretest scores. It can also be seen that the two types of instruction had nearly identical 
changes from their pretest to their posttest scores, with the e.\ception of fifth grade skills-based 
which dropped significantly more than the other three groups (see Figure 2). 

[Insert Figure 2] 

The results fix)m the recreational subtest scores indicated there were main effects of gender, F 
(1, 143)= 13.32, p< .0001, grade, F(l, 143) = 28.01, b< -0001, instruction, F(l, 143) = 
4.07, B < .045, and time of testing, F ( 1 , 1 43) = 4.49, g < .035. Results also indicated interaction 
effeas of Time of Testing X Grade, F(l, 143)= 1 9.9 1, .0001, and Grade x Type of 
Instruction x Time of Testing, F ( 1 , 1 43) = 5.06, p. < .026. 

The interaction of grade and time of testing was due to the fifth grade posttest recreational 
reading attitude scores being significantly lower than their pretest scores and were significantly 
lower than the third graders posttest scores, which were only slightly lower than their pretest 
scores. Further inspection of Figure 3 also reveals the interaction of grade and type of instruction 
and time of testing was due to the fifth grade skills-based posttest reaeational reading attitude 
scores being significantly lower than their posttest scores, as well as being significantly lower than 
the fifth grade literature-based posttest scores. It can also be seen that the third grade skills-based 
group's scores remained constant ft-om pretest to posttest (See Figure 3). 

[Insert Figure 3] 

Data obtained ftx)m the academic subtests scores revealed there were main effects of grade, F 
(1, 143)= 15.75, 2 <. 0001, and gender, F(l, 143) = 7.76, c<. 006, and interactions of Grade x 
Type of Instruction, F (1, 143) = 4.83, u < .029, and Grade x Time of Testing, F (1, 143) = 6.73, 
2< .01. 

The interaction of grade and time of testing was due to the fifth grade students' posttest 
academic reading attitude scores being significantly lower than their pretest scores and significantly 
lower than the third graders posttest scores, which was only slightly .ower than their pretest 
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scores. The intCTaction of grade and type of instruction was due to the significantly lower posttest 
scores in the fifth grade skills-based group as compared to their priest scores. The literature- 
based groups had nearly identical changes from their pretest to their posttest scores in both third 
and fifth grades. 

To distinguish which mean scores of reading attitudes were significantly different, contrasts 
[Scheffe tests (Hurford, n.d)] involving the mean pretest and posttest scores for each group were 
performed. As indicated in Table I, the difference between all pretest scores, total and both 
subtests, for all groups were not statistically significant. However, significant diffCTences existed 
brtween the third grade and fifth grade groups on the recreati(Kial reading attitude subtest and the 
total reading attitude scores (see Table I). These differences indicate third graders have more 
positive attitudes towards reading than fifth graders. A statistical difference also existed between 
the third grade literature-based group and both fifth grade groups on the academic reading attitude 
subtest, indicating fifth graders have poora* attitudes towards reading than third graders taught in a 
literature-based classroom (see Table I). 

[Insert Table I] 
CONCLUSIONS 

There is a growing movement towards the use of whole language or literature-based 
methodologies in American classrooms. Proponents of whole language believe using holistic 
strategies in the classroom will produce "lifelong" readers. In whole language classrooms, the 
focus is on the process of reading as a whole as wdl as on compreheiding literature. 

The literature suggests male students would prefer learning in a whole language environment 
because teaching strategies would favOT kinesthetic and tactile learning styles. Additionally, whole 
language advocates contend their strategies imp-ove students* attitudes towards reading. 

This study attempted to determine whether third or fifth grade male students jH*eferred 
literature-based or skills-based reading programs by measuring their attitude towards reading. 
Preferences between genders were also determined by a reading attitude survey. 

Females in both grades showed more positive attitudes towards reading than males in both 
subtests, academic and recreational reading, as well as on total reading attitude scores. Third grade 
female students in literature-based reading programs had the most positive attitudes towards 
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reading, while fifth grade male students tended to have the least positive attitudes towards reading. 
One explanation for these results might be third and fifth grade students view reading as a sex- 
appropriate role and as students grow older males view reading as a more feminine activity. 

The time of testing from pretest and posttest showed a significant difference on the total 
reading attitude scores, as well as on the subtests. In all classes, reading attitude scores dropped 
from pretest to posttest. As one may expect, attitude scores towards reading declined as the end of 
the school year approached This may be due to the faa that students beg^n to lose interest in 
school by April and May because the end of the school year was near. 

Overall, the third grade literature-based classrooms had the most positive attitudes towards 
reading, although their reading attitudes scores dropped slightly from pretest to posttest. The third 
grade skills-based classes also dropped slightly from pretest to posttest on total reading attitude and 
academic reading, but the scores remained constant ftx)m pretest to posttest on the reaeational 
reading subtest. Again, the drop in scores from pretest to posttest may be due to the lack of 
interest in school at the end of the school year. As a whole, third graders in both groups had more 
positive attitudes towards reading than fifth graders. This might suggest that third grade students 
enjoy reading more because reading and literature is still "new". 

Fifth graders had Iowa reading attitudes than third graders. Fifth graders in the literature- 
based classroom, however, had slightly b^er attitudes towards reading than the fifth grade skills- 
based classes. Initially, the fifth grade skills-based group started the school year with more 
positive attitudes towards reading than the fifth grade literature-based group. By the end of the 
school year, the fifth grade skills-based group's attitude toward reading dropped significantly 
below their pretest scores, as well as droppin,? significantly below the fifth grade literature-based 
scores. These results indicate fifth graders show a lack of interest in reading in comparison to third 
graders. However, fifth graders seem to have more positive attitudes when taught in a literature- 
based reading program than those students taught in a skills-based program. This data may 
suggest fifth graders lack interest in continued rep^ition of reading skills, but prefer the use of 
"rear literature used in meaningful context. Fifth graders may prefer to have the opportunity to 
select literature according to their intaests, such as those methods practiced in literature-based 
classrooms. 

In summary, data from this study seems to indicate a preference to the use of a literature-based 
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reading program in third and fifth grades to improve reading attitudes. Further, this study 
recommends the use of lit^ature-based reading programs with males in fifth grades to improve 
attitudes towards reading. 

FUTURE RESEARCH DIRECTIONS 

It is evident, by the literature, that past research has not addressed the effect of whole language 
pracrices in intermediate grade classes. As whole language continues to increase in popularity in 
American schools, its effects on reading attitudes and reading abilities in intermediate grades need 
to be measured. Additional research focusing on reading achievement in the intermediate grades 
should be done to determine if the use of whole language improves students' reading ability. 

It is recommended this same type of research as conducted in this study should be duplicated 
using teachers* attitudes towards reading. A comparison of the teachers* attitude towards reading 
in different reading programs would be of value in determining the effect their attitudes had on 
influencing students* reading attitudes in their classroom. 

Another recommendation for future research would be to study whether or not the teaching 
style or the reading program influences students' attitudes towards reading- From researcher 
observation, teachers in the literature-based classrooms tended to have similar teaching styles to 
each other, while teachers in the skills-based classrooms also had teaching styles similar to each 
other. Perhaps teachers with similar teaching styles prefer certain reading programs; therefore, it 
could be possible students show a preference to teaching styles instead of reading programs. 

Further research should also be conducted to control for the variable of prior reading 
instruction in earlier grades and its effect on reading attitudes. Studies measuring reading attitudes 
in entire schools that practice literature-based or skills-based reading philosophies could be used 
for subjects, testing students in each grade. Longitudinal single-subject studies using a female and 
a male subject in literature-based ot skills-based schools could be used to measure reading attitudes 
across several grade levels. 

The data from this study showed that females had more positive attitudes towards reading than 
r lales, and that third graders had more positive attitudes towards reading than fifth graders- Data 
also indicated third grade females showed a significant preference for a literature-based reading 
program. Additionally, fifth graders had less positive attitudes towards reading when a skills- 
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Literature-Based Approach 14 
based reading program was used in reading instruction. It would be of value to extend this study 
to determine if the classixwm teachers' attitudes towards reading had any influence on their 
students' reading attitudes. 
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Figure 1. Total Reading Scores 

Main Effects 

By Gender 
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Figure 2. Total Reading Scores 
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Figure 3. Recreational Reading Scores 
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Table I. Scheffe Post-Hoc 



Grade 





Third 




Fifth 




Comparison 


Literature 


Skills 


Literature 


Skills 


Pre-Recreational 


34.2 


31.4 


30.4 


31.1 


Post-Recreational 


31.6 


29.8 


25.3 


22.1 


Pre-Academic 


33.2 


30.3 


28.5 


30:7 


Post Academic 


30.4 


27.4 


23.6 


23.7 


Pre-Total 


67.0 


61.7 


58.9 


61.8 


Post-Total 


62.0 


57.3 


49.1 


45.9 



Note: All mean scores underlined indicate a nonsignificant difference between mean scores. 
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